AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	auc	0.5	0.92
	2_DecisionTree	Decision Tree	auc	0.767162	6.76
	3_Default_LightGBM	LightGBM	auc	0.793399	3.94
	4_Default_Xgboost	Xgboost	auc	0.79327	10.85
	5_Default_CatBoost	CatBoost	auc	0.797545	2.17
	6_Default_NeuralNetwork	Neural Network	auc	0.793175	4.92
	7_Default_RandomForest	Random Forest	auc	0.790046	8.49
	8_Default_ExtraTrees	Extra Trees	auc	0.78596	6.69
	9_Default_NearestNeighbors	Nearest Neighbors	auc	0.70975	3.27
the best	Ensemble	Ensemble	auc	0.798735	0.88

AutoML Performance

AutoML Performance Boxplot

Features Importance

features importance across models

Spearman Correlation of Models

models spearman correlation

Summary of 9_Default_NearestNeighbors

<< Go back

k-Nearest Neighbors (Nearest Neighbors)

n_jobs: -1
n_neighbors: 5
weights: uniform
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

2.5 seconds

Metric details

	score	threshold
logloss	1.17139	nan
auc	0.70975	nan
f1	0.686843	0.2
accuracy	0.65986	0.5
precision	0.813776	0.8
recall	0.946961	0
mcc	0.319138	0.5

Metric details with threshold from accuracy metric

	score	threshold
logloss	1.17139	nan
auc	0.70975	nan
f1	0.647308	0.5
accuracy	0.65986	0.5
precision	0.657248	0.5
recall	0.637664	0.5
mcc	0.319138	0.5

Confusion matrix (at threshold=0.5)

	Predicted as 0	Predicted as 1
Labeled as 0	1192	558
Labeled as 1	608	1070

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_CatBoost

<< Go back

CatBoost

n_jobs: -1
learning_rate: 0.1
depth: 6
rsm: 1
loss_function: Logloss
eval_metric: AUC
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

1.3 seconds

Metric details

	score	threshold
logloss	0.546035	nan
auc	0.797545	nan
f1	0.734375	0.353485
accuracy	0.723162	0.465768
precision	0.93985	0.89438
recall	1	0.0419721
mcc	0.445964	0.465768

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.546035	nan
auc	0.797545	nan
f1	0.715272	0.465768
accuracy	0.723162	0.465768
precision	0.720242	0.465768
recall	0.710369	0.465768
mcc	0.445964	0.465768

Confusion matrix (at threshold=0.465768)

	Predicted as 0	Predicted as 1
Labeled as 0	1287	463
Labeled as 1	486	1192

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 8_Default_ExtraTrees

<< Go back

Extra Trees Classifier (Extra Trees)

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: auc
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

5.7 seconds

Metric details

	score	threshold
logloss	0.565962	nan
auc	0.78596	nan
f1	0.730135	0.34611
accuracy	0.712369	0.470682
precision	0.95	0.849865
recall	1	0.134886
mcc	0.424842	0.470682

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.565962	nan
auc	0.78596	nan
f1	0.693979	0.470682
accuracy	0.712369	0.470682
precision	0.724093	0.470682
recall	0.666269	0.470682
mcc	0.424842	0.470682

Confusion matrix (at threshold=0.470682)

	Predicted as 0	Predicted as 1
Labeled as 0	1324	426
Labeled as 1	560	1118

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

n_jobs: -1
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

0.2 seconds

Metric details

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Confusion matrix (at threshold=0.440414)

	Predicted as 0	Predicted as 1
Labeled as 0	0	1750
Labeled as 1	0	1678

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_LightGBM	1
5_Default_CatBoost	2
6_Default_NeuralNetwork	1

Metric details

	score	threshold
logloss	0.546927	nan
auc	0.798735	nan
f1	0.737143	0.352068
accuracy	0.724037	0.492058
precision	0.933962	0.879807
recall	1	0.0710512
mcc	0.448467	0.534466

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.546927	nan
auc	0.798735	nan
f1	0.707121	0.492058
accuracy	0.724037	0.492058
precision	0.735825	0.492058
recall	0.680572	0.492058
mcc	0.448194	0.492058

Confusion matrix (at threshold=0.492058)

	Predicted as 0	Predicted as 1
Labeled as 0	1340	410
Labeled as 1	536	1142

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

n_jobs: -1
criterion: gini
max_depth: 3
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

5.8 seconds

Metric details

	score	threshold
logloss	0.573401	nan
auc	0.767162	nan
f1	0.719006	0.412346
accuracy	0.709743	0.412346
precision	0.834094	0.726927
recall	1	0.148592
mcc	0.423007	0.412346

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.573401	nan
auc	0.767162	nan
f1	0.719006	0.412346
accuracy	0.709743	0.412346
precision	0.683306	0.412346
recall	0.758641	0.412346
mcc	0.423007	0.412346

Confusion matrix (at threshold=0.412346)

	Predicted as 0	Predicted as 1
Labeled as 0	1160	590
Labeled as 1	405	1273

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 3_Default_LightGBM

<< Go back

LightGBM

n_jobs: -1
objective: binary
num_leaves: 63
learning_rate: 0.05
feature_fraction: 0.9
bagging_fraction: 0.9
min_data_in_leaf: 10
metric: auc
custom_eval_metric_name: None
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

3.0 seconds

Metric details

	score	threshold
logloss	0.56427	nan
auc	0.793399	nan
f1	0.734248	0.360764
accuracy	0.72112	0.493773
precision	0.916667	0.798934
recall	1	0.162908
mcc	0.443142	0.493773

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.56427	nan
auc	0.793399	nan
f1	0.699371	0.493773
accuracy	0.72112	0.493773
precision	0.740346	0.493773
recall	0.662694	0.493773
mcc	0.443142	0.493773

Confusion matrix (at threshold=0.493773)

	Predicted as 0	Predicted as 1
Labeled as 0	1360	390
Labeled as 1	566	1112

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

<< Go back

Summary of 6_Default_NeuralNetwork

<< Go back

Neural Network

n_jobs: -1
dense_1_size: 32
dense_2_size: 16
learning_rate: 0.05
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

4.1 seconds

Metric details

	score	threshold
logloss	0.552003	nan
auc	0.793175	nan
f1	0.730374	0.298259
accuracy	0.722287	0.526779
precision	0.951923	0.931535
recall	1	0.0168114
mcc	0.444867	0.526779

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.552003	nan
auc	0.793175	nan
f1	0.70398	0.526779
accuracy	0.722287	0.526779
precision	0.736021	0.526779
recall	0.674613	0.526779
mcc	0.444867	0.526779

Confusion matrix (at threshold=0.526779)

	Predicted as 0	Predicted as 1
Labeled as 0	1344	406
Labeled as 1	546	1132

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 7_Default_RandomForest

<< Go back

Random Forest

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: auc
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

7.5 seconds

Metric details

	score	threshold
logloss	0.556767	nan
auc	0.790046	nan
f1	0.733082	0.326399
accuracy	0.721704	0.462388
precision	0.941176	0.872696
recall	1	0.098249
mcc	0.443013	0.462388

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.556767	nan
auc	0.790046	nan
f1	0.711084	0.462388
accuracy	0.721704	0.462388
precision	0.722906	0.462388
recall	0.699642	0.462388
mcc	0.443013	0.462388

Confusion matrix (at threshold=0.462388)

	Predicted as 0	Predicted as 1
Labeled as 0	1300	450
Labeled as 1	504	1174

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

n_jobs: -1
objective: binary:logistic
eta: 0.075
max_depth: 6
min_child_weight: 1
subsample: 1.0
colsample_bytree: 1.0
eval_metric: auc
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

auc

Training time

9.9 seconds

Metric details

	score	threshold
logloss	0.551109	nan
auc	0.79327	nan
f1	0.734109	0.36918
accuracy	0.721412	0.419541
precision	0.916256	0.877082
recall	1	0.0539589
mcc	0.444362	0.415532

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.551109	nan
auc	0.79327	nan
f1	0.723749	0.419541
accuracy	0.721412	0.419541
precision	0.703204	0.419541
recall	0.74553	0.419541
mcc	0.444038	0.419541

Confusion matrix (at threshold=0.419541)

	Predicted as 0	Predicted as 1
Labeled as 0	1222	528
Labeled as 1	427	1251